The present invention relates to a program and method for facilitating decision-making, and more particularly, the present invention relates to a computer program, system, and method that accepts inputs that bear on a decision in a structured format from a plurality of sources and arranges the inputs in a hierarchical structure that permits an evaluation of the inputs as they relate to the decision.
Decision-makers in businesses and government agencies are routinely called upon as part of their jobs to make decisions with the intention of achieving desired outcomes. One example is that of a regulatory decision-maker in a government agency charged with establishing policies and regulations with the intention of achieving a desired result. Often the assessment of the alternatives, and likely outcome of a proposed policy or regulation, depends upon relevant scientific research and knowledge. If the intent is to develop policies that are science-based, as opposed to personal or political, then use of sound scientific grounds in the decision-making process improves the probability that the desired outcome will result from a policy or regulation.
Unfortunately, despite the intentions of the decision-makers, the policy and regulation making process is not always tightly linked to relevant scientific evidence. One possible reason for this is that gaps exist between scientists and policy makers, where it is often difficult or impossible for a policy maker to understand, and effectively use, all the relevant scientific evidence that is available.
In general, public testimony and anecdotal evidence suggest that many policy makers/regulators would like to draw upon all pertinent scientific evidence, but simply cannot. In some cases, not enough research has been completed; and in cases where a large volume of research does exist (often with varying degrees of uncertainty, and debate over interpretation), it is often organized in a way that is inscrutable to the typical policy maker, who is not a scientist. "Science assessment", the collection and assessment of scientific information and evidence in support of policy decision making, does not in itself solve the inscrutability problem, because science assessments do not necessarily document and organize the scientific arguments in a way that nonscientists can understand.
The problem, more precisely, is that it is often difficult for the policy makers to understand how content specialty experts, scientists and engineers, arrive at their judgments, and what to believe when two experts give contradictory answers. This gives rise to the inscrutability problem. In other words, there is a plethora of research, often reaching somewhat different conclusions, and it is organized and presented in a way that is difficult for a policy maker to understand in relation to the policy issue that needs resolution. Also, the science or technology assessment and review process often takes too long to accomplish because experts are inefficiently networked. The problem is further compounded by lack of an efficient structure for eliciting relatively complete, precise and accurate claims, rebuttals, and counterclaims directly from the experts, in a way that is practical and efficient. After the assessment, the details of how a science assessment question was answered and why are often lost, because of incomplete documentation as the details or elements of the arguments are not recorded and archived properly.
Policy makers may go through large science and technology assessments that require multiple experts in order to fully cover the range of issues surrounding the policy question. The problem begins with the need to assess an option. Experts are tapped to provide expert opinions and supporting evidence, but the experts are often located in various locations around the country or world. The process is time consuming, because the experts need to be contacted and provided with explanations of what portion of their expertise is needed in order to provide useful input to the science or technology assessment. The expert's response is usually a summary, without the details of how they came to that assessment. The grounds and backing for the specific assessment that is given by the expert are usually not documented well if at all. If two experts give contradictory assessments, it is usually difficult for the policy maker to understand how their arguments compare, such as the different grounds and backing each bring to bear on their claims. The policy maker attempts to choose between them, by deciding which argument, or hypothesis, has the greatest degree of support. When information is missing, the policy maker may have to go back to the experts to get clarifications. This process of considering of the science or technology, identifying missing information and going back to the experts, can take weeks or longer. As a result of this, the process is prone to omission of important details.
Decision and policy makers often find it difficult to understand and consider the respective expert's science or technology assessments and how they relate to the given policy question because the assessment may omit many of the details, such as the grounds and backing upon which conclusions were drawn and the confidence that the expert has in those conclusions. It is difficult for policy makers to understand and assess the arguments put forth by experts who have conflicting expert opinions. In addition, it is difficult for the policy makers to see hidden bias on the part of the expert for or against an option. Further, it is difficult to see how the arguments presented by the expert relate to the original science or technology assessment question.
Experts intuitively assess the degree of support that the scientific evidence brings to bear on their claims or hypotheses, but this assessment is usually not explicitly stated and recorded. Peer review is may be required but there is, very often, no means whereby the peer review group can systematically review the assessment through use of all the grounds and backing that led to the assessment conclusions. The group usually just does not have the information structured in such a way that they can systematically see the grounds and backing that has been brought forth by the expert making the claim, therefore making it difficult to assess the degree of support that should be associated with the hypothesis or claim.
After the decision or policy makers collect the respective pieces of the assessment and bring them together in context of the policy question, they often ask another group to provide a peer review. If an expert does not agree with a claim made in the assessment, he or she may provide a rebuttal, and in some cases a counterclaim. Here again, there is lack of structure and systematic means of engaging in this process. It is time consuming, often taking many months before this step can be completed.
Once the expert's review has been provided, the policy maker should be able to understand and properly consider the expert's claims and counterclaims. If the assessment is believed to be adequate, as judged by those conducting the assessment, then it needs to be documented for use by policy makers. If it is felt to be not adequate, the assessment process needs to continue through another cycle. This can be a time consuming and costly process and efforts to make it more efficient and timely would be desirable.
There are at least six common factors in large science assessments which make it hard for policy decision makers to understand (inscrutability). First, there is too much information (information overload) and insufficient structuring of the facts and evidence presented by the various experts. Second, often it is not clear how the data are brought to bear on what has been claimed because arguments are not explicitly stated and are difficult to understand in the context of the policy question: (e.g., there is inadequate linkage of the available and relevant scientific data and information to what has been claimed, it is hard for policy decision makers to get information; and experts and policy decision makers are working together from different disciplines and often different geographical locations). Third, each discipline involved in the assessment has its own field dependent knowledge and field specific vocabularies. Fourth, there are different (analytical) frames of reference without clear structure to how they relate. Fifth, different experts present contradictory claims. Sixth, there are issues outside of scientific inquiry influence the assessment, such as social norms, political goals, economics (profits) and different value systems, which are a necessary and inseparable part of the assessment.
These factors commonly effect science assessments because, there is the need to bring the arguments put forth by experts or scientists to bear on policy decisions, and these arguments are often contradictory and conflicting. These science assessments are typically monumental tasks which are plagued with this gap between experts and policy decision makers, failed use of information and resulting poor policy decisions. The need for structure to address the above list of contributing factors is common to the science assessment, in general.
An example of the type of policy decision-making process that utilizes a large complex science assessment science assessment is the program being conducted by the U.S. Global Change initiative. Experts from several countries and multiple scientific disciplines are working with policy decision makers in an effort to get international regulation of pollutants and practices that are adversely effecting the worlds changing climate and ecosystems. Global change encompasses the full range of natural and human-induced changes in the Earth's environment and includes changes in the global environment (including alterations in climate, land productivity, oceans or other water resources, atmospheric chemistry, and ecological systems) that may alter the capacity of the Earth to sustain life. The goal of the global change science assessment is to support national and international policy decision making. This science assessment has major consequences for society; and if properly understood and considered by national and international policy decision makers, the consequences can be positive.
The Global Change science assessment is only one of many. Another example is acid rain science assessment. There are thousands of experts in the fields of human health, ecology, material, visibility, sociology, and economics involved in the science surrounding acid rain policy making.
Still another example of this type of science assessment is the assessment being conducted for pest management policy decision making. The pest management (e.g. pesticide) policy making activities performed by the U.S. Department of Agriculture and U.S. Environmental Protection Agency require science assessment because the effectiveness of pest management is based on plant biology, entomology, chemistry, toxicology and several other disciplines. Pest management has become dependent on pesticides, which is a major concern to U.S. federal and state policy decision makers, because most pesticides pose risks to human health and the environment.
Government regulation of pesticides is primarily done by the U.S. Environmental Protection Agency (EPA), however other agencies of the U.S. government are involved including the U.S. Department of Agriculture (USDA). In general, EPA is responsible for identifying pesticides that cause cancer in humans or animal as specified by the Federal Insecticide Fungicide and Rhodenticide Act (FIFRA). Under the Delaney Clause of FIFRA, companies need to show that pesticides do not cause cancer in humans or animals. Under this law, if any dosage of a pesticide causes cancer, it must be regulated.
Science assessment is conducted as part of the pest management policy decision making process to assess the risk, such as the human dietary risk from residues that result from use of a pesticide on food. For example, a pressing concern is the discovery that pesticides get into infants' diets. EPA is conducting science assessment to find acceptable levels (tolerances) of these toxicants that infants can be exposed to with a reasonable risk. It is not likely that regulations will result in the absolute elimination of pesticides in diets, but a special coordinated effort is underway by EPA, USDA and the Department of Human Health and Safety (HHS) to safeguard the health of infants and children since they are believed to be at greater risk than adults. These assessments are supported by scientific findings; experts, primarily from companies that profit from the sales of pesticides, are required to provide scientific findings and data to EPA (test results) as supporting evidence that a pesticide they want to register for use does not cause cancer.
USDA contributes to the policy decision making process under a working relationship with EPA These science assessments identify viable alternatives to the pesticides that EPA has or is considering regulating. Under this arrangement USDA and EPA cooperate (1) to find viable alternatives to pesticides under regulatory review, (2) to identify areas of priority need, and (3) to award research grants to worthy proposals in those areas of need. Science assessment is required for all three.
Logistics present a problem in the policy-making process. Federal policy decision makers primarily reside primarily in Washington, D.C. and the experts primarily reside in Land Grant Universities in the states and territories. There are only a few USDA and EPA policy makers who have a limited amount of time and resources for science assessment in the policy development process, but they are required to tap hundreds of experts from many different disciplines in the Land Grant Universities.
The USDA and EPA policy decision makers may also find it difficult to understand and properly consider claims being put forth by the various experts. A pesticide regulation may take away a pesticide that is the current and perhaps only means of controlling a pest, and this can cause devastating pest damage to the commodity and result in huge financial losses for the growers. The scope of the required science assessment can be enormous considering the number of commodities that must be considered, the number of pests that effect each commodity, and the number of pest management tactics (chemical and non-chemical) that are plausible for controlling each pest on each commodity. For even a single decision, these decision-makers consider expertise from experts in multiple disciplines concerning the viability of a single alternative. These policy decision makers may also consider whether crop/pest areas are priorities for government funded research, a difficult task considering that there are hundreds of types of commodities grown in the U.S. and each commodity has multiple pest problems (insects, rodents, fungi, weeds and other types of pests).
Another complexity of these science assessments is that the hundreds of pest management tactics have varying degrees of effectiveness. Furthermore, pests adapt for survival through genetic resistance, usually causing an effective pest management tactic to eventually lose its effectiveness, and the timing and practices of pest management effect the rate at which genetic resistance develops in the pests. For example, a pesticide used alone for several years may lose its effectiveness due to genetic resistance in the near-term, but used at reduced rates in combination with other chemical or non-chemical tactics the genetic resistance may not occur until many years later.
Contradictory arguments provided by the experts significantly increase the magnitude of the problem. For example, in the 1995 Pesticide Review List Survey, there were many cases where experts had conflicting reports on the effectiveness of alternative pest management tactics, and it was not clear in many cases whether these reports were based on sound scientific data or not. There are two aspects that stand out for improvement here: first, better data, such as yield and quality data, need to be collected, and second, data need to be brought to bear (linked) to the arguments of effectiveness that are being put forth.
Still another aspect that causes additional complexity is that a claim put forth at one point in time is subject to rebuttal at some point in the future, such as when the effectiveness of an alternative changes, or when an expert attempts to replicate a field study in which effectiveness data are collected, but the results are contradictory.
In this example the gaps between these policy decision makers and experts may cause the policy decision makers to make decisions without considering all relevant arguments: when the experts do not clearly and concisely convey the important information, and make their arguments explicit, then the policy decision makers are not able to understand them. This problem was evident when policy decision makers were considering which crop/pest problems should be included in the 1995 Pest Management Alternatives for Farmer RFP. They depended on the experts for the sound scientific evidence to build an arguments for or against including a particular crop/pest problem in the RFP by doing science assessment involving respective experts. The policy decision makers surveyed the experts for information and received responses back, but the grounds for the arguments and how the arguments were warranted were not documented: (1) the experts were not tapped effectively, (2) the policy decision makers did not understand (inscrutability), (3) consequently they did not properly consider all the relevant information that was known about the alternative pest management alternatives, and (4) it would have taken too long to do a comprehensive peer review--even though they used the system of experts within the Land Grant University system that USDA had constructed for this very purpose. To bridge this gap, these decision makers should have arguments explicitly stated by the experts that are easy to understand and fully consider in the policy decision making process.
To recap, the magnitude of pest damage from policy decision making makes the stakes high, either to agricultural production or to human health and environment. In the absence of an effective means of tapping the experts, and understanding and considering the arguments, the decisions may just be politically driven, without basis in sound objective science. There are many examples like the ones above where information exists, but failure to use it leads to poor policies. Policy decision makers typically want sound scientific evidence to bear on policy decision, but are frustrated by this gap between them and the experts.
In general, prior art efforts to use scientific assessments to make sound policy decisions have failed when the scope of the assessment encompasses a large number of sources. Prior efforts have not been effective at capturing the arguments being put forth by the experts. They have not captured the grounds (data with references) upon which conflicting claims of experts are based, and they have not properly lined claims to supporting data.
Accordingly, it is an object of the present invention to provide an improved system and method that can facilitate decision-making, particularly when the decision-making utilizes for support a large number of sources. It is a further object, to provide a method wherein scientific assessment can be effectively utilized in a decision-making process.